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PROBLEM 

The  Cost  Plus  Award  Fee  contract  (CPAF)  Is  an  appealing  concept  because 
it  allows  procurement  managers  to  use  judgement  In  rewarding  performance. 

It  has  recently  been  alleged,  however,  that  a high  and  relatively  constant 
percentage  of  the  award  fee  is  always  paid.  If  this  Is  true,  the  effective- 
ness of  CPAF  contracts  relative  to  other  types  Is  In  doubt,  objc^t^s  st-Jy 

^08<tfffi?ES-~0f-4H€~STU0Y  

— 1.  Establish  current  level  of  award  fee  payments  within  Army  contracts. 

2.  Investigate  procedures  to  determine  what  award  fee  to  pay. 

3.  Accumulate  evidence  Indicating  the  motivational  effectiveness  of 
cost- pi  us -award -fee  contracts. 

4.  Evaluate  current  guidance  for  the  use  of  CPAF  contracts. 

5.  Recommt nd  change  In  use  or  guidance,  if  appropriate. 

D /METHODOLOGY 

he  data  b*ie  for  this  study  consisted  of  (1)  19  completed  DARCOM  CPAF 
contracts  in  the  1970-1975  timeframe  (approximately  1/2  of /the  population), 

(2)  26  active  CPAF  contracts  (about  one-third  of  the  dlscdrnable  population), 
and  (3)  response^  from  experienced  CPAF  user's  In  each  Major  Subordinate 
Command  (MSC)  In/ DARCOM. 

nferences  toward  the  above -objectives  were  drawn  from  analyses  of 
award  fee  payment  level,  the  relationship  of  award  fee  payment  to  various 
other  variables  (e.g.,  fixed  fee  base),  time  and  command  trends,  and 
qualitative  assessments. 

CONCLUSIONS  AND  RECOMMENDA^PHS 


The  current  mean  level  of  award  fee  payments  Is  83%  with  a standard  devia- 
tion of  6%.  This  average  and  spread  are  consistent  with  the  NASA  and  DOD 
award  fee  rating  system.  Extensive  Investigation  reveals  that  Individual 
awards  vary  extensively  over  a wide  range  and  that  It  Is  not  true  that  a 
high  and  relatively  constant  percentage  of  the  award  fee  Is  always  paid. 

Award  fee  payments  have  not  been  automatic;  they  vary  significantly  In 
relative  size  with  performance  Indicating  that  the  CPAF  contract  type  Is 
effective.  Award  fee  payments  have  not  varied  significantly  among  commands 
nor  with  time.  None  of  the  other  external  factors  examined  appear  to 
influence  the  award  fee  amount.  However,  some  problems  remain.  Including  one 
defect  In  the  evaluation  procedure  that  results  In  a bias  In  award  fee  pay- 
ments under  certain  circumstances.  The  procedures  used  for  determining  the 
amount  of  award  fee  to  pay  are  comprehensive  and  appear  to  be  working  In 
the  field  as  evidenced  by  award  fee  results  as  well  as  direct  Inspection  and 
evaluation  of  the  procedures.  The  technique  Is  necessarily  very  flexible 
and  utilizes  different  philosophies  In  different  applications.  This  creates 
the  need  for  an  Interchange  of  Ideas  and  experience  among  technical  personnel 
^ var1ous  commands.  It  is  recommended  that  research  continue  and  that 
DARCOM  strengthen  Its  policy  on  award  fee  contracting  to  facilitate  the 
Interchange  of  experience  and  research. 
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CHAPTER  I 


INTRODUCTION 

A.  BACKGROUND 

The  Cost  Plus  Award  Fee  (CPAF)  contract  Is  a cost  reimbursement  contract 
which  has  been  used  by  the  DOD  and  NASA  since  1962.  It  was  developed  to 
provide  a means  of  establishing  contractual  Incentives  where  the  development 
of  firm  objectives  for  cost,  technical  and  management  performance  could 
not  be  made.  Recently  two  major  expressions  of  concern  within  the  Army 
have  arisen  concerning  the  use  of  CPAF  contracts.  One  Involves  the  adequacy 
of  guidance  documents  for  structuring  the  contract  and  In  evaluating  con- 
tractor performance.  The  other  related  concern  Involves  the  appropriateness 
and  motivational  effectiveness  of  CPAF  contracts  in  accomplishing  their 
Intended  purpose;  l.e.,  contractor  motivation.  To  address  these  Issues 
this  study  evaluates  a unique  feature  of  CPAF  contracting;  l.e.,  award  fee 
payments,  and  evaluates  the  effectiveness  of  the  fee  awarded  as  a motivational 
factor  In  contract  performance. 

B.  PROBLEM 

" > 

The  award  fee  Is  an  appealing  concept  since  It  Is  a means  for  apiplydng 

Incentives  In  those  cost  relnfcursement  contracts  that  do  not  readily  lend 
themselves  to  finite  measurements  of  performance  required  ffl*  structuring' 


Incentive  contracts.  However,  a shadow  of  doubt  has  been  cast  over  the 
effectiveness  of  the  award  fee  as  a motivational  device.  It  has  been  alleged 
that  a high  and  relatively  constant  percentage  of  the  award  fee  Is  always 
paid.  If  this  Is  uniformly  true  then  It  would  appear,  on  the  surface  at 
least,  that  the  cost-plus-award-fee  (CPAF)  contract  differs  little  In  opera- 
tion from  a cos t-plus^ fixed- fee  (CPFF)  contract.  In  fact.  If  practice  re- 
veals a lack  of  award  fee  motivation,  then  the  higher  administrative  cost 
of  the  award  fee  feature  would  make  the  CPAF  contract  less  desirable  than  the 
CPFF  contract. 

In  view  of  the  above,  an  assessment  and  evaluation  of  award  fee  pay- 
ments under  Army  contracts  Is  in  order. 

c.  msjim 

The  objeettvee  of  this  study  or*  to:  (1)  establish  the  current  level  of 
award  foe  payments  within  Army  contracts,  (11)  Investigate  the  various  pro- 
cedures employed  to  determine  what  proportion  of  the  available  award  fee  to 
award,  (111)  accumulate  evidence  both  positive  and  negative  Indicating  the 
motivational  effect! venose  (or  the  lack  of  motivational  effectiveness)  of 
Army  cost-plus-award-fee  contracts,  (1v)  evaluate  current  guidance  regarding 
the  uoo  of  CPAF  contracts.,  and  (v)  recommend  changes  In  CPAF  usage  or 
guidance,  if  appropriate, 
o.  WnNOOQLW 

The  research  methods  employed  during  the  study  were  directed  toward 
obtaining  a quantitative  assessment  of  award  fee  contracts  as  used  In  the 
Army.  The  procedure  Involves  establishing  a theory  reflecting  what  award 


fee  payments  Mould  be  expected  to  be  under  the  hypothesis  that  they  are 
effective  and  under  the  alternate  hypothesis  that  they  are  Ineffective.  Then 
the  actual  level  of  award  fee  payments  Is  computed  and  compared  to  the 
theoretical  models.  This  raises  questions  that  must  be  answered  before  a 
rational  assessment  can  be  made.  Answers  to  these  questions  are  pursued  in 
two  ways.  The  current  guidance  and  current  usage  In  determining  what 
awards  should  be  made  is  Investigated  and  statistical  analysis  of  the  data 
base  Is  employed  to  assess  the  effectiveness  with  which  the  system  operates. 

A comparison  of  how  Individual  awards  are  distributed  versus  how  they  would 
be  expected  to  be  distributed  under  both  hypotheses  Is  Investigated  to 
establish  the  structure  necessary  to  give  the  previous  assessments  additional 
meaning.  Conclusions  are  then  developed  as  the  facts  warrant. 

E.  SOURCES  OF  INFORMATION 

Information  used  In  the  analysis  was  derived  from  two  sources.  The 
first  source  consisted  of  a data  base  of  completed  CPAF  contracts  and  a 
second  more  limited  data  base  from  ongoing  contracts.  Both  of  these  data 
bases  are  described  at  the  beginning  of  Chapter  III. 

The  second  type  of  Information  used  In  the  study  Is  qualitative  In 
nature  and  was  obtained  from  a review  of  contract  files  and  from  interviews 
with  those  personnel  experienced  In  the  use  of  CPAF  contracts.  Personal 
Interviews  were  conducted  with  procurement  policy  staffs,  contracting  officers. 


contract  specialists  and  members  of  award  fee  boards.  The  extensive 
experience  and  knowledge  of  these  experts  were  most  helpful  In  Interpreting 
the  data  obtained  from  individual  contract  files  and  In  providing  examples 
of  the  effect  of  award  fee  on  the  motivation  of  contractors. 

F.  ORGANIZATION  OF  REPORT 

Chapter  II  contains  a brief  discussion  of  the  application,  limitations 
and  guidance  documents  pertaining  to  CPAF  contracting  and  also  contains 
an  evaluation  of  the  guidance.  Some  problems  are  also  surfaced  here.  In 
Chapter  Ilia  brief  description  of  the  data  base  (a  more  complete  description 
is  contained  in  Appendix  A)  is  presented  followed  by  an  analysis  of  the 
level  of  award  fee  payments.  Chapter  III  also  provides  observations  and 
statistical  analysis  directed  toward  resolving  some  of  the  problems  encountered 
to  provide  sufficient  information  to  evaluate  the  effectiveness  of  the  pro- 
cedure and  its  application  in  field  use.  Chapter  IV  presents  the  conclusions 
and  recommendations  of  the  study. 


CHAPTER  II 


AWARD  FEE  CONTRACT  GUIDANCE, 
THEORY  AND  FIELD  APPLICATION 


A.  GENERAL 


This  chapter  briefly  reviews  the  current  policy  and  theory  of  award 
fee  contracting.  The  theory  is  then  analyzed  as  the  basis  to  investi- 
gate field  application.  Then  field  experience  is  evaluated  In  light  of 
these  findings  with  the  view  of  separating  the  problems  of  an  unresolved 
nature  from  those  that  have  been  successfully  solved.  The  results  are 
then  related  back  to  the  theory  and  guidance. 

B.  REVIEW  OF  CURRENT  GUIDANCE 

This  section  contains  a brief  outline  of  the  application  and 
limitation  of  CPAF  contracts  and  a stannary  review  of  the  appropriate 
guidance  documentation  In  the  area. 

1 . Application  and  Limitation  of  CPAF  Contracts 

The  CPAF  contract  provides  a means  of  applying  incentives  In  cost- 
reimbursement  contracts  which  are  not  susceptible  to  development  of 
scaled  objectives  for  cost,  technical,  and  management  performance.  A 
CPAF  contract  Includes:  (1)  an  estimated  cost,  (2)  a base  fee  which 

does  not  vary  with  performance  (the  base  fee  may  be  zero),  (3)  an 
award  fee  which  may  be  earned  In  whole  or  In  part  or  not  at  all  with  the 
amount  actually  awarded  to  be  determined  by  a subjective  evaluation  of 
performance  measured  against  the  performance  criteria  set  forth  In 


. 
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the  contract,  (4)  a maximum  fee  (base  fee  plus  award  fee  pool),  (5) 
performance  criteria,  and  (6)  the  fee  payment  plan.  CPAF  contracts 
are  appropriately  used  for: 

(I)  level  of  effort  contracts  for  performance  of  services 
where  mission  feasibility  Is  established  but  measurement 
of  achievement  must  be  by  subjective  evaluation  rather 
than  objective  measurement  and 

(II)  work  which  would  have  been  placed  under  another 
type  of  contract  If  the  performance  objectives  could 

be  expressed  In  advance  by  definite  milestones,  targets 
or  goals  susceptible  of  measuring  actual  performance.' 

CPAF  contracts  are  not  used: 

(I)  . . . as  an  administrative  technique  to  avoid  CPFF 
contracts  when  the  criteria  for  CPFF  contracts  apply, 
nor  shall  a CPAF  contract  be  used  to  avoid  the  effort 
of  establishing  objective  targets  so  as  to  make 
feasible  the  use  of  a CPIF  type  contract. 

(II) .  . . where  the  contract  amount,  period  of 
performance  or  the  benefits  expected  are  Insufficient 
to  warrant  the  additional  administrative  effort  or 
cost. 

(III) .  . . for  procurements  categorized  as  either 
Engineering  Development  or  Operational  System 
Development  (see  4-101 (a) (6)  a (7))  which  have 
undergone  contract  definition,  except  that  where  It 
may  be  more  advantageous  to  do  so.  It  may  be  used  In 
these  categories  for  Individual  procurements,  ancillary 
to  the  development  of  a major  weapon  system  or  equipment, 
where  the  purpose  of  the  procurement  Is  clearly  to 
determine  or  solve  specific  problems  associated  with 

the  major  weapon  system  or  equipment.1  2 


1 ASPR  3-405. 5(b)  dated  1 Oct  75. 

2ASPR  3-405. 5(g)  dated  1 Oct  75. 
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More  detailed  definitions  as  well  as  applications  and  limitations 
on  the  use  of  CPAF  contracts  are  contained  in  various  procurement 
guidance  documents  of  the  DOD  and  NASA.  For  In-depth  Information  on 
CPAF  contracting  the  reader  Is  referred  to  these  documents  which 
are  now  briefly  summarized. 

2.  Guidance  Documents 

a.  Armed  Services  Procurement  Regulation.  The  primary  and  most 
succinct  source  of  guidance  on  the  use  of  CPAF  contracts  Is  contained 

In  ASPR  3-405.5.  Excerpts  from  this  guidance  are  cited  above  for 
general  Information  on  CPAF  contracting.  More  detailed  Information, 
e.g.,  amount  of  base  and  maximum  fee  and  evaluation  considerations  and 
criteria,  are  Included  In  the  ASPR  and  are  necessary  reading  for  those 
personnel  preparing  for  the  solicitation  and  award  of  CPAF  contracts. 

b.  POD  and  NASA  Incentive  Contracting  Guide.  The  DOD  and  NASA 
Incentive  Contracting  Guide  dated  October  1969  contains  one  chapter  on 
CPAF  contracts.  However  this  guide  only  covers  the  subject  In  general 
terms  and  from  a philosophical  viewpoint.  It  provides  little  Information 
not  contained  In  the  above  cited  ASPR  reference. 

c.  NASA  Cost  Plus  Award  Fee  Contracting  Guide.  The  NASA  Cost 
Plus  Award  Fee  Contracting  Guide  dated  August  1967  Is  the  most  com- 
prehensive Government  document  on  CPAF  contracting.  Structuring  con- 
tracts and  establishing  evaluation  criteria  as  well  as  organizing  for 
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the  administration  of  contracts  are  very  effectively  covered.  Also 
examples  of  award  fees  and  a study  of  CPAF  contractual  "theory"  are 
presented.  Unfortunately  the  "theory"  Is  on  a philosophical  plane 
and  provides  little  help  toward  application  other  than  to  very  adequately 
summarize  the  concepts  Involved.  It  provides  a compendium  of  the  often 
conflicting  economic,  psychological  and  sociological  explanations  of 
Individual  motivation.  It  does  provide  an  extensive  enumeration  of  the 
many  factors  and  effects  that  may  operate  and  therefore  should  be 
considered  In  this  very  nebulous  area. 

d.  Army  and  PARCOH  Regulations.  Supplemental  regulations  on 
the  use  of  CPAF  contracts  are  not  contained  In  the  Army  Procurement 
Procedures.  However,  DARCQM  Circular  No.  715*9-75  dated  10  September 
1975  provides  command  procurement  Instructions  on  the  designation  of 
award  fed  determining  officials  for  CPAF  contracts. 

e.  Major  Subordinate  Command  Regulations.  Except  for  coverage 
In  the  ASPR,  there  Is  a general  dearth  of  enlightening  Information 

on  CPAF  contracting  at  the  000,  Army  and  OiARCON  levels.  As  a result, 
most  MSC's  of  0ARC0M  have  Issued  their  own  comprehensive  regulations 
and  Instructions.  Although  most  of  these  regulations  and  Instructions 
describe  circumstances  which  are  unique  to  a particular  Installation, 
and  therefore  reflect  to  some  extent  individual  command  philosophy 
much  of  the  information  Is  dupllcatory  and  similar  to  the  more  general 
information  and  procedures  contained  In  the  NASA  Cost  Plus  Award  Fee 
Contracting  Guide. 
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c.  ANALYSIS  OF  CPAF  CONTRACTING  "THEORY" 

The  purpose  of  the  award  fee  provision  Is  t 6 motivate  a contractor 

to  stimulate  outstanding  performance  In  areas  that  cannot  be  objectively 

measured.  The  primary  vehicle  Is  profit  translated  Into  terms  of  an 

award  fee  payment  that  is  In  turn  related  to  a subjective  evaluation  of 

(nonquantlflable)  performance  In  selected  areas: 

Profit  generally.  Is  the  basic  motive  of  business  enterprise. 

Both  the  Government  and  Its  defense  contractors  should 
be  concerned  with  harnessing  this  motive  to  work  for  the 
effective  and  economical  contract  performance  required  In 
the  Interest  of  national  defense.  To  this  end,  the  parties 
should  seek  to  negotiate  and  use  the  contract  type  best 
calculated  to  stimulate  outstanding  performance.  The 
objective  should  be  to  Insure  that  outstanding  effective 
and  economical  performance  Is  met  by  high  profits,  mediocre 
performance  by  mediocre  profits,  and  poor  performance 
by  low  profits  or  losses.  The  proper  application  of  these 
objectives  on  a contract  by  contract  basis  should  normally 
result  In  a range  of  profit  rates.3 

...  In  certain  cases.  It  may  be  desirable  to  motivate  and 
reward  a contractor  for  management  performance  over  and 
above  that  which  can  be  objectively  measured  and  Incentlvlzed 
under  other  forms  of  government  contracts.  For  example, 
logistics  support,  quality,  timeliness,  cooperation.  Ingenuity, 
and  cost  effectiveness  are  areas  under  the  control  of  management 
which  may  be  susceptible  only  to  subjective  measurement  and 
evaluation.  In  such  cases,  the  "award  amount"  portion  of  the 
fee  applicable  to  the  CPAF  contract  Is  an  Ideal  method  for 
Incorporation  of  these  additional  Incentives  Into  government 
contracts.  . . 4 


3ASPR  3-401 (b)(1)  dated  1 October  1975. 
4ASPR  3.405.5(h)  dated  1 October  1975. 


It  is  concluded  therefore  that  performance  Is  to  be  equated  to 
profit  oh  the  award  fee  payment  in  the  following  way: 


Performance 


Profit 


OUTSTANDING  = HIGH 

MEDIOCRE  = MEDIOCRE 

POOR  = LOW  OR  NEGATIVE 

The  negative  profit  alternative  is  not  a part  of  the  award  fee 

feature  unless  the  complicating  economic  concept  of  opportunity  cost 
or  potential  profit  foregone,  is  taken  into  account.  This  approach  was 
avoided  for  the  purposes  of  this  study. 

The  key  question  is  obviously  exactly  how  Is  this  relation  to  be 
established.  An  example  of  a criteria  for  evaluating  contractor  performances 
Is  given  in  ASPR  3.405. 5(y)(1)  where  performance  Is  ranked  from  Excellent 
down  to  Submarginal  and  then  equated  to  a numerical  measurement  scale 
of  100  to  0.  Since  the  relevant  variables  making  up  the  quality  of 
performance,  by  definition,  cannot  be  measured  exactly  In  the  award  fee 
case  the  scores  provided  In  the  example  have  to  refer  to  contractors 
in  general  when  thinking  In  terms  of  absolutes.  In  terms  of  award  fee 
evaluation  It  provides  a conceptual  framework  for  a point  of  reference 
but  it  can  only  be  applied  literally  In  a relative  (subjective)  sense 
and  not  as  a literal  linear  transformation  (absolute  sense).  It  Is  In 
fact  a flexible  guide. 


* 


...  The  number  of  criteria  used  and  the  requirements 
which  are  represented  will  differ  widely  from  one 
contract  to  another.  Therefore,  when  determining 
criteria  and  rating  plans  the  usng  activity  should 
be  flexible  and  select  a plan  wh’ch  will  motivate  the  5 
contractor  in  a positive  way  to  Improve  performance.  . . 

The  suggested  evaluation  then  is  a flexible,  relative  aid  to  equate  per- 
formance (as  subjectively  evaluated  against  Internal  goals  unique  to  that 
contract)  to  an  award  fee  payment  (a  direct  representation  of  profit.)  NASA 
has  suggested  a different  plan  for  making  this  conversion  using  some  of 
the  same  words  but  associated  with  different  levels  on  a numerical  "point" 
scale.  The  relationship  of  these  two  systems  Is  .;hown  in  Table  1. 

The  NASA  plan  is  based  on  the  perception  of  a set  of  utilities  differing 
somewhat  from  those  implied  in  the  000  suggested  evaluation.  NASA  defines 
an  overall  contractor  performance  evaluation  score  of  80  as  corresponding 
to  what  is  normally  expected  of  a contractor  and  also  defines  this  point  on 
the  scale  as  the  median  numerical  rating.  An  award  fee  of  50%  Is  then 
equated  to  the  median  contractor  score  of  80  points.  The  full  scale  of 
potential  award  fees  is  developed  as  shown  In  Table  1 based  on  their  stated 
assunption  of  linear  deviations  from  the  median. 

It  follows  from  the  above  that  an  "average"  contractor  doing  what  is 
t normally  expected  would  earn  an  award  fee  of  50%  under  this  system. 

| 

i — 

ASPR  3-405. 5(a) 


Successful  efforts  to  motivate  a contractor  toward  Improved  performance 
results  in  an  award  fee  payment  above  50%  with  the  desired  goal  of  superior 
performance  resulting  In  award  fee  payments  within  the  90t-100t  range. 

The  DOD  plan  does  not  define  a specific  point  on  the  contractor  performance 
evaluation  scale  reflecting  what  Is  "normally  expected"  of  a contractor. 
However,  if  the  criteria  used  by  NASA  (The  mid-point  of  their  "good"  rating) 
is  applied  to  the  DOD  scale,  as  Is  often  done  In  the  field,  a rating  of  75 
points  emerges  as  the  normal  standard.  These  differing  views  of  what  Is 
the  expected  standard  reference  point  to  use  In  evaluating  a specific  con- 
tract contribute  to  various  Interpretations  In  the  field  and  widely  differing 

award  fee  scales,  (see  Table  2) 

% 

C.  EVALUATION  OF  FIELD  EXPERIENCE  IN  RATING 
1 . The  Effective  Scale  Used  In  Evaluation 

Often  In  the  award  determining  procedure,  factors  are  quantified  on 
a rating  scale  of  zero  to  100.  Then  some  minimum  level  of  performance  Is 
required  (usually  60  to  70  points  total  score)  to  achieve  any  award.  The 
apparently  obvious  fact  that  the  scale  of  0 to  100  Is  being  used  to  determine 
the  amount  of  the  award  Is  misleading.  That  part  of  the  scale  below  the 
cut  off  point,  say  70  points.  Is  used  only  to  determine  If  any  award  fee 
Is  to  be  paid.  The  scale  above  the  cut  off  point.  In  this  case  the  30 
points  between  70  and  100,  becomes  the  effective  range  of  the  scale  for 
determining  the  amount  of  the  award. 


I 


TABLE  1 


DOD  CONTRACTOR  PERFORMANCE  EVALUATION  AND 
NASA  CPAF  SUPPORT  CONTRACT  RATING  SYSTEM 


DOD  Contractor  Performance 

NASA  Rating  System  for  CPAF  Support  Contracts 

Evaluation 

Adjective 

Points  * 

Specific 

Evaluation 

Score 

Rating 

% Award 

- 

Adjective 

100 

100 

Superior 

99 

Superior 

97.5 

Superior  Minus 

92.5 

Excellent 

96 

90 

95 

87.5 

• 

Excellent 

85 

Excellent  Plus 

91 

• 

90 

. 

75 

• 

Excellent 

Very  Good 

k 

• 

67.5 

Excellent 

85 

65 

1, 

Good 

62.5 

(Median) 

NASA  Expected 

60 

Good  Plus 

80 

Performance 

50 

Good 

■ 

Good 

76 

• 

42.5 

Good  Minus 

75 

40 

• 

• 

• 

37.5 

70 

Satisfactory 

35 

Satisfactory 

Plus 

69 

• 

Satisfactory 

• 

17.5 

Satisfactory 

Marginal 

66 

65 

15 

Minus 

• 

• 

12.5 

61 

Fair 

10 

2.5 

Fair  Plus 

60 

0 

Fair 

Sotoaarglnal 

• 

• 

0 

~ .. 

13 
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2 . Widely  Differing  Conversion  Methods 

Each  award  fee  contract  is  an  individually  tailored  document 
designed  to  meet  a specific  and  unique  situation.  It  is  necessary  that  the 
procedure  retain  its  flexibility  to  permit  exact  "tailoring"  to  meet  the 
differing  situations  of  each  contract.  However,  It  is  questionable  if 
some  of  the  diversity  encountered  in  the  field  is  actually  necessary  or 
desirable.  The  motivation  behind  some  of  the  more  unusual  conversion 
schemes  often  seems  to  be  an  attempt  to  engineer  the  award  fee  toward  some 
predetermined  level  rather  than  to  enhance  performance  on  the  critical  areas 
of  the  contract.  The  procedure  almost  universally  employed  to  determine  the 
appropriate  award  fee  is  to  build  up  a composite  score  from  many  factors  and 
subfactors  and  then  to  convert  this  score  into  a percentage  of  the  award 
fee.  In  some  cases  the  score  and  the  percentage  of  the  award  are  the  same. 
In  others  they  differ  widely.  A score  of  99  (out  of  a possible  100  points) 
for  example  would  result  in  award  of  the  potential  (maximum)  award  In  some 
cases.  In  others  It  would  result  In  percentages  as  shown  In  Table  2. 

There  may  sometimes  be  valid  reasons  for  such  a procedure  but  It  gives  the 
appearance  of  reducing  the  award  below  what  was  apparently  earned.  A more 
direct  approach  would  be  a straightforward  conversion  with  the  evaluation 
contained  within  the  score  system,  not  added  later  by  conversion. 

3.  Detailed  Micro  Evaluations 

The  concept  of  an  award  fee  contract  Is  that  It  Is  used  In  special 
Instances  when  It  Is  Impossible  to  objectively  quantify  the  performance 
elements;  hence,  a subjective  evaluation  Is  necessary.  Yet,  In  practice. 


TABLE  2 


I ' 

SCHEMES  FOR  CONVERTING  PERFORMANCE  SCORE  TO  AWARD  FEE 


I 


the  evaluation  on  most  award  fee  contracts  is  broken  down  Into  many 
factors  and  subfactors  and  sometimes  contain  a further  breakdown  Into 
Individual  elements  - all  of  which  are  scored  numerically.  There  are  some 
circumstances  when  this  Is  reasonable  such  as  when  this  type  of  contract 
is  used  as  a transition  from  a previous  fixed  fee  to  a future  Incentive 
contract.  In  such  a case  one  of  the  purposes  of  the  award  fee  feature  Is 
to  develop  such  a quantitative  evaluation  system.  There  are  perhaps  other 
special  conditions  that  warrant  such  an  evaluation  but  the  prevalence  of  the 
practice  raises  the  question;  If  the  evaluation  process  Is  really  subject 
to  such  detailed  and  accurate  measurement  why  Isn't  It  already  an  Incentive 
contract  ( CP IF)?  The  fragmentation  of  objectives  by  such  a detailed  break- 
down may  very  well  dilute  the  Incentive  feature.  Contracts  have  been 
scored  on  such  a low  level  of  detail  that  In  one  case  a single  factor  was 
worth  $28.  On  the  other  hand,  notable  success  has  been  obtained  In  some 
contracts  that  concentrate  the  award  on  a few,  four  or  five,  really 
Important  factors. 

4.  The  Averaging  Effect  In  Detailed  Evaluations 

Another  effect  Involved  In  contracts  with  detailed  breakdown  of 
the  award  fee  results  from  the  mechanics  of  determining  the  fee.  Consider 
a detailed  evaluation  procedure.  An  Individual  element  Is  evaluated  over 
several  months,  usually  three  or  six,  by  each  of  several  COR's.  First  each 
COR  averages  his  monthly  scores,  which  may  be  different,  then  the  Individual 
COR's  evaluations  are  averaged.  The  elements  are  then  weighted  and  combined 


Into  factors  that  make  up  a total  score.  This  averaging  effect  tends  to 
bring  the  scores  toward  the  center  of  the  portion  of  the  scale  that  Is 
effectively  used.  The  chances  of  a contractor  obtaining  a perfect  score 
are  slim  Indeed.  The  only  way  this  can  be  accomplished  Is  for  each  Individual 
micro-evaluation  to  be  uniformly  high,  an  unlikely  event  under  most  circum- 
stances. Suppose  the  contractor  did  a perfect  job  and  one  COR  for  some 
reason  erroneously  rated  him  low.  The  magnitude  of  the  error  will  be  mitigated 
to  some  extent  by  averaging  with  many  other  factors  all  of  which  we  hope  are 
correct.  However,  the  error  will  never  be  “averaged  out"  and  he  will  not 
receive  a perfect  score  to  match  his  perfect  performance,  unless  of  course 
someone  gives  him  a "better  than  perfect"  score,  and  this  doesn't  happen  In 
practice. 

This  can  be  easily  demonstrated  with  a simple  example.  Table  III  shows 
an  example  of  a hypothetical  performance  evaluation  containing  an  erroneous 
rating  on  a "perfect"  contractor.  The  contractor  fully  met  all  criteria  on 
each  rating  Item  and  should  have  received  100  ratings  for  each  sub-category 
resulting  In  a total  weighted  rating  of  100.  However,  the  contractor  was 
erroneously  given  an  80  rating  under  criteria  A-l  (shown  In  parenthesis). 

The  effect  of  this  downgrade  Is  carried  through  the  roll  up,  as  shown  by 
the  figures  in  parenthesis,  with  no  chance  of  being  corrected  by  compensating 
errors.  The  final  result  Is  a total  score  of  97.6  points.  This  results  In 
the  award  fee  being  reduced  from  Its  correct  value  of  100*  by  a factor  de- 
termined from  the  particular  conversion  scheme  used.  The  key  point  Is  that 
there  Is  no  possibility  of  other  errors  compensating  for  the  original  rating 
error  and  "averaging"  It  out  of  the  rating  system. 
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within  procurement  circles.  Successful  award  fee  contracts  have  been 
administered  when  the  evaluation  plan  has  been  given  to  the  contractor  and 
also  when  It  was  not  given  to  him.  Communicating  results  of  the  evaluation 
to  the  contractor  Is  another  area  that  often  causes  difficulty.  The  con- 
tractor needs  to  know  his  areas  of  less  than  superior  performance  as  well 
as  the  areas  where  he  excells  If  he  Is  to  Improve  In  the  future.  It  is 
important  to  keep  the  ultimate  objective  In  mind;  maintain  a superior  per- 
formance and  thereby  earn  the  maximum  award.  Establishing  the  priority  of 
requirements  and  developing  the  plan  itself  are  continuing  difficulties  that 
require  continuing  attention.  However,  these  are  problems  that  were  trouble- 
some at  the  beginning  of  award  fee  contracting  and  are  now  manageable  with 
the  development  of  field  procedures  and  the  accumulation  of  experience. 

E.  SUMMARY  OF  CPAF  EVALUATION  THEORY  AND  PRACTICE 

The  award  fee  provision  Is  Intended  to  motivate  a contractor  to  achieve 
outstanding  performance  In  areas  that  cannot  be  objectively  measured. 

Each  contract  Is  Individually  tallQred  to  meet  a specific  and  unique  set  of 
circumstances.  Performance  which  Is  subjectively  evaluated  against  unique 
Internal  goals  within  a contract  must  be  converted  to  a numerical  award  fee 
payment  representing  profit  to  the  contractor.  The  exact  methods  of  the 
conversion  from  the  subjective  evaluation  to  the  numerical  reward  Is  detailed 
In  the  rating  scheme  contained  within  the  evaluation  plan.  The  realistic 
development  and  Implementation  of  this  rating  system  Is  one  of  the  more 
troublesome  problems  In  putting  the  award  fee  theory  Into  practice. 


In  actual  practice  subjective  evaluations  are  universally  converted 
to  numeric  ratings  by  the  evaluation  and  rolled  up  Into  an  overall  numeric 
rating.  This,  In  turn  Is  mechanically  converted  Into  a recommended  award 
fee  subject  to  later  review  and  adjustment.  The  application  of  this  process 
creates  some  problems  In  the  field. 

In  devising  a rating  plan  for  an  Individual  contract,  contracting 
officers  need  a benchmark  on  an  absolute  scale  Indicating  what  Is  normally 
expected  of  a contractor  to  provide  a point  of  reference.  The  apparent 
difference  between  the  DOD  and  NASA  suggested  standard  causes  confusion  and 
leads  to  an  unnecessary  wide  range  In  conversion  methods. 

The  process  of  evaluation  as  now  used  In  the  field  has  an  inherent  bias 
built  into  It  at  both  the  high  and  low  extremes  of  the  performance  scale  that 
tends  to  pull  extreme  awards  toward  the  middle.  The  process  of  very  detailed 
micro  evaluations  often  tend  to  dilute  the  award  fee  to  a point  where  its 
effectiveness  In  motivating  contractor  performance  Is  questioned. 


CHAPTER  III 

ANALYSIS  OF  AWARD  FEE  PAYMENT  DATA 


A.  GENERAL 

In  this  chapter  statistical  analysis  of  data  from  two  sources  Is 
presented.  First  the  data  bases  are  described;  then  the  level  of  current 
award  fee  payments  Is  Investigated.  Next  several  areas  are  Investigated 
to  assist  In  determining  the  effectiveness  of  the  fee  determining  system. 
This  Is  necessary  to  provide  a framework  In  which  to  Interpret  the 
award  fee  level. 

B.  DATA  BASE 

During  the  course  of  the  study  effort  two  data  bases  on  CPAF 
contracting  were  obtained.  The  first  consisted  of  a sample  of  relevant 
Information  extracted  from  the  files  of  19  completed  CPAF  contracts. 

This  Is  the  base  for  most  of  the  analysis  which  follows.  The  second 
sample  of  the  existing  data  for  26  on-going  contracts  Is  used  to  corroborate 
some  of  the  findings  from  the  analysis  of  completed  contract  data.  In 
both  cases  only  "pure"  CPAF  contracts  were  sampled.  Those  with  unusual 
or  complicating  provisions  such  as  "catch  up"  or  "balloon"  provisions 
and  those  using  the  award  fee  feature  in  conjunction  with  other  Incentive 
provisions  were  excluded.  The  samples  were  selected  In  this  manner 
In  order  to  Isolate  the  effects  of  the  award  fee  behavior  as  much  as 
possible.  Most  of  these  contracts  were  for  engineering  and  technical 
services  and  all  were  Initiated  since  October  1969.  Although  both 
samples  are  small  the  completed  contract  data  represented  over  one  half 


of  all  completed  CPAF  contracts  and  the  on-going  data  represents  over  one- 
third  of  all  on-going  CPAF  contracts  for  the  time  period  as  reported  on  the 
DD  Form  350's.  A more  complete  description  of  the  sample  data  from  the 
consisted  contracts  is  contained  In  Appendix  A. 

C.  LEVEL  OF  AWARD  FEE  PAYMENTS 

One  of  the  primary  objectives  of  this  study  was  to  establish  the 
current  level  of  award  fee  payments  within  the  Army.  This  estimate  will 
be  based  on  analysis  of  the  primary  data  base  of  completed  contracts.  But, 
before  procedlng  directly  to  the  objective  It  is  necessary  to  investigate 
the  possible  effect  of  two  potentially  confounding  factors;  the  Influence 
of  time  and  the  comnand  where  the  contract  was  awarded. 

1.  Effect  of  Conroand  on  the  Level  of  Award  Fee  Payments 

The  procedures  employed  to  detect  differences  between  commands 
Is  to  compare  the  mean  (average)  values  within  each  command.  If  data 
on  all  contracts  for  each  command  were  available  this  could  be  readily 
done  and  It  could  be  ascertained  at  once  that  there  Is  or  Is  not  a difference. 
But  since  such  data  Is  not  available  within  the  practical  constraints  of 
this  study,  the  true  value  for  all  contracts  within  a command  must  be 
estimated  from  the  data.  And  since  different  samples  of  the  same  size 
would  contain  different  combinations  of  all  possible  contracts,  the 
resulting  estimates  of  the  true  values  would  differ  slightly.  It  is 


necessary  therefore  to  apply  some  decision  rule  to  Insure  that  the  differences 
observed  In  the  sample  are  not  likely  due  to  sampling  error  (chance)  alone 
but  represent  some  other  real  differences.  This  procedure  Is  formally 
embodied  In  the  standard  statistical  method  of  Analysis  of  Variance  (ANOVA) 
which  Is  used  here  to  test  the  level  of  award  fee  payments  for  differences 
between  commands,  and  later  for  differences  due  to  time.  Because  of  the 
small  sample  size,  both  of  these  tests  are  limited  in  the  statistical 
inferences  that  may  be  drawn  from  them.  They  are  used  only  to  determine  If 
the  data  may  be  pooled  in  computing  the  award  fee  payment  level.  Because 
of  the  variability  of  the  data  shown  In  the  frequency  distribution.  Figure 
1,  the  variances  were  tested  In  both  cases  and  found  to  be  homogeneous. 

The  results  of  the  ANOVA  test  for  command  differences  In  award  fee  level  Is 
now  presented. 


TABLE  4 

ANALYSIS  OF  VARIANCE  FOR  CONTRACT  AWARD  FEES  BETWEEN  COMMANDS 


Source  of  Variation 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 

Square 

F 

Ratio 

Test* 

Value 

Between  Coanands 

109.34 

2 

54.67 

1.56 

3.63 

Within  Coamands 

560.86 

16 

35.05 

Total 

670.20 

18 

*F(.05,  2,  16) 
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On  the  basis  of  this  test  It  Is  concluded  that  for  the  purpose  of  pooling 
the  data  there  are  no  detectable  differences  between  commands  In  the 
level  of  award  fee  payments  for  completed  contracts.  One  coamand  has 
only  three  contracts  and  another  only  five;  therefore,  no  other  conclusions 
are  made  at  this  time. 

2.  Effects  of  Time  on  Award  Fee  Payments 

In  order  to  test  the  effects  of  time  on  the  level  of  award  fee 
payments  the  data  are  grouped  by  year  of  completion.  The  data  for  1972 
and  prior  years  Is  placed  In  one  group  because  of  the  small  sample  and 
scarcity  of  data.  The  results  follows. 

TABLE  5 

ANALYSIS  OF  VARIANCE  FOR  CONTRACT  AWARD  FEES  BETWEEN  YEARS 


Source  of  Variation 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 

Square 

F 

Ratio 

Test* 

VAlue 

» 

Between  years 

59.78 

3 

19.93 

0.47 

3.34 

Within  years 

590.40 

14 

42.17 

Total 

650.18 

17 

*F(.05,  3,  14) 

On  the  basis  of  this  test  It  Is  concluded  that  for  the  purpose  of  pooling 
data  there  are  no  detectable  differences  between  years  In  the  level  of 
award  fee  payments  for  completed  contracts. 


3.  Estimate  of  the  Average  Award  Fee  Payment 

In  light  of  the  fact  that  no  evidence  was  shown  of  any  effect  of 
time  of  contract  completion,  or  command  of  contract  execution,  on  the 
level  of  award  fees  paid.  It  Is  concluded  that  we  hava  a homogeneous  data 
base  for  the  purpose  of  establishing  the  level  of  award  fee  payments 
within  the  Army.  The  best  point  estimate  of  the  current  level  of  award 
fee  payments  as  a percentage  of  the  maxlmian  available  award  fee  Is  given 
by  the  sample  mean.  This  value  is  83%  (Table  6).  Because  this  value  is 
computed  from  a sample  that  does  not  Include  all  award  fee  contracts  that 
have  been  completed  during  the  time  span  of  the  data  base,  two  range  estimates 
are  also  given  with  different  confidence  levels,  based  on  the  sample 
standard  deviation  of  5.93%.  The  ranges  are  80.25%  to  85.70%,  for  a 95% 
confidence  level  and  78.90%  to  87.06%  for  a 99%  confidence  level.  The 
statement  Is  made  that  the  true  value  of  award  fee  payments  for  all  contracts 
falls  within  the  range  specified  with  the  associated  degree  of  confidence 
(95%  or  99%). 

TABLE  6 CURRENT  LEVEL  OF  AMARD 
FEE  PAYMENTS  ESTIMATED  FROM  SAMPLE  DATA 


Confidence  Level 

Value  Range 

N/A  (Point  Estimate) 

82.98% 

95% 

80.26%-85.70% 

99% 

78.90%- 87. 06 
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FREQUENCY 


FIGURE  1 


FREQUENCY  DISTRIBUTION  OF  THE  LEVEL  OF  AWARD  FEE  CONTRACTS 


PAYMENTS 


This  brings  up  the  question  of  what  the  level  of  award  fee  payments  should 
be.  If  the  system  Is  working  It  should  be  high,  for  It  Is  the  payment 
associated  with  the  desired  level  of  performance.  However,  high  pay- 
ments do  not  necessarily  mean  that  performance  Is  high  unless  the  evalu- 
ation system  Is  working  properly.  This  Is  the  subject  of  the  next  section 
D.  EFFECTIVENESS  OF  THE  AWARD  FEE  DETERMINING  SYSTEM 


The  most  desirable  way  of  evaluating  the  award  fee  determining 
system  Is  In  terms  of  over  all  cost  effectiveness.  However,  the  practical 
problems  discussed  briefly  In  section  2 below  render  such  an  approach 
Infeasible  for  this  research.  The  procedure  followed  Is  to  Investigate  the 
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predictability  of  the  award  fee  payments  and  their  variability.  Predict- 
ability Is  associated  with  the  relationship  of  award  fee  payments  to  other 
selected  factors  and  with  their  variability.  The  variability  of  Individual 
award  fae  payments  Is  Investigated  both  between  different  contracts  and 
within  Individual  contracts.  Analysis  of  the  on-going  contract  data  pro- 
vides further  evidence  of  the  diversity  of  Individual  awards.  The  results 
of  these  Investigations  are  related  In  the  sections  which  follow  to  what 
might  reasonably  be  expected  In  order  to  evaluate  the  effectiveness  of  the 
award  fee  determining  system. 

2.  Cost  Effectiveness 

An  attempt  was  made  in  this  study  to  assess  the  cost  effectiveness 
of  award  fee  contracts.  It  was  quickly  learned  that  It  was  Impossible 
to  Isolate  the  actailnl strati ve  costs  with  a reasonable  amount  of  resources. 
This  already  had  been  attempted  by  some  contracting  officers  who  found  that 
the  day-to-day  adnlnlstratlve  activities  were  so  intertwined  that  they  defy 
delineation  within  any  reasonable  level  of  effort.  The  differences  between 
contracts  Is  tremendous.  Estimates  for  preparation  of  evaluation  board 
folders  varies  from  three  days  to  three  weeks.  Board  members  said  that 
they  could  not  accurately  estimate  the  preparation  that  they  made  for  the 
evaluation  boards.  In  view  of  these  difficulties  the  attespt  to  compute 
administrative  costs  was  discontinued.  One  of  the  determinations  that  must 
be  made  before  the  CPAF  type  contract  Is  selected  Is  that  they  are  expected 
to  be  cost  effective. 


The  CPAF  contract  shall  not  be  used  where  the  contract 
amount,  period  of  performance  or  the  benefits  expected 
are  Insufficient  to  warrant  the  additional  administrative 
effort  or  cost.^ 

In  reviewing  the  contract  files  and  talking  with  contracting  officers  and 
contract  specialists  the  conclusion  was  reached  that  CPAP  contracts  can  be 
cost  effective.  The  technique  has  been  used  to  successfully  obtain  goals 
that  were  not  reached  under  other  contracting  techniques.  The  technique 
Is  not  universally  applicable.  However,  for  selected  procurements  experience 
has  shown  that  the  benefits  obtained  fully  justify  the  extra  costs  Involved. 
The  remainder  of  this  chapter  addresses  the  effectiveness  side  of  the  cost- 
effectiveness  question. 

3.  Relationship  of  Award  Fee  Payments  to  Selected  Other  Factors 
Award  fee  payments  have  been  challenged  on  the  basis  that  they 
do  not  motivate  the  contractor  and  are  therefore  Ineffective;  It  Is  said 
that  payment  depends  on:  size  of  contract  (dollar  value),  or  absolute 

size  of  the  potential  award  fee*  In  dollars  (magnitude)  or  relative  size  of 
the  potential  award  (percentage  of  estimated  contract  cost),  among  others. 

The  award  fee  payments  should  depend  primarily  on  contractor  performance 
and  be  related  to  other  factors  only  to  the  extent  that  contractor  per- 
formance Is  Itself  related  to  them. 


]ASPR  3-405. 5(g)(11)  dated  1 Oct  75. 
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It  Is  also  possible  that  the  level  of  award  fee  payments  may  be 
related  to  the  fee  structure  itself.  If  this  is  true  then  some  insight 
into  how  to  improve  the  motivational  effectiveness  of  CPAF  contracts  may 
be  gained  from  investigation  of  this  phenomenon.  These  questions  Are 
addressed  in  this  section  of  the  report.  The  method  employed  for  this 
analysis  is  the  standard  statistical  technique  of  simple  linear  regression 
analysis  with  the  model: 

Y * a + bX 

where 

Y - the  level  of  award  fee  payment 

X - the  Independent  variable  described  In  each  regression. 

The  procedure  used  to  test  the  significance  of  the  regression  Is  an 
F statistic  computed  as  follows: 

F(',  N-2.  N-2)  " 

which  Is  compared  with  a predetermined  test  value,  here  F with  alpha  > .05 
(5%  significance  level)  and  17  degrees  of  freedom;  l.e.,  F^  * (.05,  17,  17) 
If  the  calculated  F value  exceeds  the  test  value  the  association  of  the  two 

variables  Is  considered  significant;  not  likely  to  have  occurred  by  chance. 
Otherwise,  the  null  hypothesis  of  no  correlation  cannot  be  rejected  and  the 
association  Is  termed  non>s1gn1f1cant;  likely  to  have  occurred  by  chance. 


in  i n<|  the  <* 1 1 «•«: L of  v.irioir.  factors  on  the*  award  fen  payment 

in  the  Mih'.ne.L  ion*,  that,  follow  the  methods  briefly  outlined  above  are 
applied  at  two  points  in  the  life  of  the  contract.  (As  stated  previously 
the  independent  variable,  the  award  fee  payment,  can  only  be  determined  at 
:he  completion  of  the  contract).  The  relationship  of  the  dependent  variables 
that  are  alleged  to  have  caused  some  of  the  variation  in  the  award  fee 
payments  are  investigated  at  contract  initiation  to  develop  relationships 
as  the  fee  structure  was  planned,  and  at  contract  completion  to  assess  how 
it  actually  happened.  It  has  already  been  shown  that  there  is  a shift  in 
fee  structure  between  these  two  points.  Therefore  the  consistency  of  these 
two  related  estimates  is  also  of  Interest. 

a.  Effect  of  Size  of  Contract  on  Award  Fee  Payment.  To  test  the 
proposition  that  the  size  of  a contract  exerts  a positive  Influence  on 
the  percentage  of  available  award  fee  paid,  the  estimated  cost  of  initial 
contracts  and  completed  contracts  are  compared  with  the  corresponding  award 
fees,  yielding  the  following  results: 

TABLE  7 

REGRESSION  OF  AWARD  FEE  PAYMENT  ON  CONTRACT  SIZE 


Cost  Variable 

a 

b 

r 

F 

Results* 

Initial  Contracts 

31.72 

.02 

.26 

1.70 

non-slgnlf leant 

Completed  Contracts 

81.78 

.01 

.21 

1.53 

non-significant 

*.05  Level  of  Significance 
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The  null  hypothesis  of  no  correlation  cannot  be  rejected  In  either 
case.  The  argument  previously  stated  is  judged  false.  Therefore,  based 
on  analysis  of  the  sample  data,  the  size  of  the  contract  (either  initial 
cost,  or  completed  contract  cost)  is  not  an  indicator  of  the  expected  award 
fee  payment. 


b.  Effect  of  Magnitude  of  Award  Fee  on  Award  Fee  Payment.  The 


supposition  has  been  made  that  the  absolute  value  of  the  maximum  poten- 


tial award  fee  (award  fee  pool)  has  a positive  influence  on  the  award  fee 
payment.  The  argument  goes  that  contractors  will  tend  to  "work  harder"  for 
the  bigger  potential  awards  and  Ignore  the  smaller  awards,  a very  logical 
assumption.  This  is  closely  related  to  the  argument  above  but  Incorporates 
an  additional  factor,  the  percentage  of  costs  that  constitute  the  award 
fee.  This  hypothesis  was  tested  using  the  dollar  value  of  the  award  fee 
pool  at  contract  Initiation  and  at  contract  completion. 

TABLE  8 


REGRESSION  OF  AWARD  FEE  PAYMENT  ON  SIZE  OF  AWARD  FEE  POOL 


The  null  hypothesis  of  no  correlation  cannot  be  rejected  in  either 
case.  Therefore  analysis  of  the  data  does  not  provide  any  evidence  of 
a relationship  between  the  percentage  of  the  available  award  fee  pool  that 
is  actually  paid  and  the  site  in  dollars  of  the  pool  either  It  Initiation 
of  the  contract  or  at  contract  completion!  On  first  Inspection  this  result 
was  surprising,  but  reinspection  of  the  data  gave  no  reason  to  doubt  the 
lack  of  correlation;  and  other  factors  among  the  contracts  which  might 
influence  the  size  or  earned  amount  of  the  award  fee  were  not  constant. 

c.  Effects  of  Relative  Size  of  Potential  Award  Fee  on  Award  Fee 
Payment.  The  argument  has  been  presented  that  the  percentage  of  potential 
award  fee  earned  (award  fee  payment)  would  Increase  as  the  award  fee  pool 
becomes  a larger  percent  of  the  estimated  contract  cost;  l.e. , the  larger 
the  award  fee  pool  in  relation  to  estimated  cost,  the  more  Incentive  it 
would  provide  the  contractor  to  perform.  A counter  argument  maintains  that 
the  more  complex,  higher  risk  contracts  would  have  the  higher  award  fee 
and  this  factor  would  not  influence  the  percentage  of  award  fee  earned.  The 
relative  size  hypothesis  was  tested  using  the  award  fee  pool  as  a per- 
centage of  cost  at  contract  initiation  and  contract  completion. 


TABLC  9 


REGRESSION  OF  AWARD  FEE  ON  RELATIVE  SIZE  OF  AWARD  FEE  POOL 


Relative  Award  Fee 

a 

b 

r 

F 

Results* 

Initial  Contracts 

83.59 

-0.12 

-0.02 

1.04 

non-significant 

Completed  Contracts 

87.12 

-0.61 

-0.12 

1.27 

non-significant 

*.05  Level  of  Significance 


The  null  hypothesis  of  no  correlation  cannot  be  rejected  In  either 
case.  Therefore  analysis  of  the  data  does  not  provide  any  evidence  of 
a relationship  between  the  percentage  of  available  award  fee  pool  that 
was  actually  paid  and  the  relative  size  of  the  pool  in  relation  to 
estimated  cost. 

d . Effect  of  Magnitude  of  Fixed  Fee  on  Award  Fee  Payment.  The 
assumption  is  sometimes  made  that  the  base  fee  of  an  award  fee  contract 
acts  as  a cushion  making  the  contractor  less  Interested  In  earning  the 
award  fee.  If  this  hypothesis  Is  true  then  It  follows  that  there  is  a 
negative  correlation  between  the  absolute  size  of  the  base  fee  and 
percentage  of  available  award  fee  earned.  This  hypothesis  Is  tested  by 
the  regression  analysis  below. 

TABLE  10 

REGRESSION  OF  AWARD  FEE  PAYMENT  ON  MAGNITUDE  OF  FIXED  FEE 


Magnitude  Fixed  Fee 

a 

b 

r 

F 

Resul ts 

Initial  Contracts 

87.00 

-0.44 

-0.52 

3.17* 

significant 

Completed  Contracts 

84.45 

-0.07 

-0.32 

1.94 

non-significant 

*.05  Level  of  significance}  F "2.28 

(.05.  17.  17) 


The  null  hypothesis  of  no  correlation  between  award  fee  payment  and 
magnitude  of  fixed  fee  Is  rejected  on  the  basis  of  data  on  Initial 
contracts.  However,  the  null  hypothesis  does  hold  up  for  completed 
contracts.  This  Inconsistency  1$  troublesome.  Close  Inspection  of  the 
raw  data  Indicate  some  slight  shifts  In  the  fixed  fee  during  execution 
and  major  shifts  In  the  absolute  value  of  fixed  fee  primarily  as  a result 
of  frequent  major  changes  In  estimated  contract  costs.  Interpretation  of 
this  result  Is  deferred  until  the  Investigation  of  the  relationship 
between  award  fee  payments  and  relative  (X)  fixed  fee  which  follows. 

e.  Regression  of  Award  Fee  Payment  on  Relative  Size  (X)  of 
Fixed  Fee.  The  argument  of  subsection  d above  can  be  restated  with  the 
relationship  between  award  fee  and  relative  size  (percent  of  cost)  of  the 
fixed  fee.  This  relationship  Is  tested  below. 

TABLE  11 

REGRESSION  OF  AWARD  FEE  ON  RELATIVE  SIZE  OF  FIXED  FEE 




Relative  Size  FF  a b 

r F Results 

Initial  Contracts  88.66  -2.55  -( 

).46  2.71*  significant 

Completed  Contracts  86.41  -2.46  -0.45  2.64*  significant 


The  null  hypothesis  of  no  correlation  Is  rejected  In  both  cases. 

This  clearly  Indicates  the  Important  result  that  award  fee  payments  are 
negatively  related  to  the  fixed  fee  when  expressed  as  a percentage  of 
expected  cost  and  that  this  relationship  Is  consistent  with  conditions 
at  both  the  beginning  and  at  the  end  of  the  contract.  Combining  this 
result  with  result  d above  It  Is  evident  that  a relationship  does  exist 
between  the  award  fee  payment  and  the  fixed  fee  and  that  this  relationship 
is  consistent  at  both  contract  Initiation  and  contract  completion  when  the 
fixed  fee  Is  expressed  as  a percentage  of  cost.  This  result  suggests  the 
desirability  of  nvestlgatlng  the  fee  ratio  and  Its  possible  effect  on 
the  award  fee  payment.  Further  discussion  will  follow  after  that 
analysis. 

f.  Effect  of  Award  Total  Fee  Ratio  on  the  Award  Fee  Payments. 

It  seems  reasonable  to  assume  that  as  more  of  the  total  fee  Is  allocated 
to  the  award  fee  and  less  to  the  fixed  fee  the  contractor  would  be  more 

highly  motivated  and  would  likely  earn  a higher  percentage  of  the  available 

\ 

award  fee.  This  hypothesis  Is  tested  In  the  following  regression  using 
the  ratio  of  award  fee  to  total  fee  for  the  Independent  variable. 

TABLE  12 

REGRESSION  OF  AMARD  FEE  PAID  ON  AWARD  TOTAL  FEE  RATIO 


Fee  Ratio 

a 

b 

r 

F 

Results* 

Initial  Contracts 

63.99 

0.24 

0.43 

2.51* 

significant 

Completed  Contracts 

63.23 

0.25 

0.46 

2.70* 

significant 

*.05  Level  of  Significance;  F ■ 2.28 

(.05,  17,17) 
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The  null  Hypothesis  of  no  correlation  Is  rejected  In  both  cases. 

A positive  relationship  exists  that  Is  consistent  for  the  fee  structure 
at  contract  Initiation  and  contract  completion  between  the  amount  of 
aval  labia  award  fee  paid  and  the  ratio  of  the  potential  award  fee  to 
the  total  fee.  The  higher  the  award  fee  relative  to  the  total  fee,  and 
therefore  the  smaller  the  fixed  fee,  the  larger  the  percent  of  potential 
award  fee  that  Is  paid.  This  result  Is  also  consistent  and  similar  to 
the  closely  related  analysis  In  subsection  e which  It  strongly  supports. 

The  results  obtained  In  the  various  regression  analyses  are  drawn  together 

* 

In  the  next  subsection. 

g.  Summary  and  Results  of  Regression  Analysis.  The  arguments 
that  the  award  fee  payment  is  Influenced  by  size  of  contract  or  by  size 
of  potential  award  fee,  either  In  absolute  or  relative  terms,  have 
been  shown  to  be  unsupportable  and  were  judged  false.  The  award  fee 
payment  acts  as  though  It  Is  Independent  of  these  factors  as  Indeed  It 
should  be  If  the  award  fee  concept  Is  working  correctly.  The  award  fee 
payment  should  be  a function  of  performance. 

However,  the  award  fee  payment  does  exhibit  a weak  relationship 
with  the  fee  structure  as  expressed  by  the  ratio  of  the  award  fee  to  the 
total  fee.  In  view  of  the  facts  that;  the  total  fee  Is  composed  of  the 
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fixed  fee  and  award  fee,  and  since  the  award  fee  has  been  shown  unrelated 

to  the  award  fee  payments,  and  In  view  of  the  close  similarity  of  results 

and  similar  consistency  between  the  analysis  of  e and  f above  It  becomes 

obvious  that  the  fixed  fee  Is  the  variable  actually  associated  with  the 

award  fee  payment.  The  most  consistent  and  therefore  reliable  and  useful 

measure  revealing  the  effect  of  the  fixed  fee  Is  the  fixed  fee  expressed 

as  a percentage  of  cost.  The  relationship  developed  for  this  variable 

for  completed  contracts  In  e above  will  be  used  below  to  draw  statistical 

Inferences  generalizing  this  relationship. 

h.  Inferences  from  the  Fixed  Fee  Award  Fee  Payment  Relationship. 

As  indicated  previously  the  linear  relationships  between  the  relative 

fixed  fee  and  award  fee  payment  Is  statistically  significant  at  the  5% 

level  of  significance.  It  Is  now  of  Interest  to  know  how  much  of  the 

variation  In  award  fee  payments  can  be  attributed  to  differences  In  the 

2 

fixed  fees.  Such  an  estimate  Is  given  by  100  • r where  r Is  the  Pearson 
product  - moment  coefficient  of  correlation.  Such  calculations  for  the 
two  relationships  developed  In  subsection  e yield  results  of  21.16%  for 
Initial  contracts  and  20.25%  for  completed  contracts.  Therefore,  It  Is 
concluded  that  for  our  sample,  variations  In  percent  of  fixed  fee  accounts 
for  or  explains  sllqhtly  over  20%  of  the  observed  variation  In  award  fee 
payments.  The  relationships  are  expressed  graphically  In  Figure  2. 

It  would  be  of  obvious  Interest  to  place  confidence  Intervals  on  the 
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FIGURE  2 RELATIONSHIP  OF  AWARD  FEE  PAYMENT  TO 
FIXED  FEE  ESTIMATED  FOR  SAMPLE  DATA* 


- 


CONTRACT  COST 


* 

This  relationship  Is  statistically  significant  at  the  95* 
confidence  level  and  explains  about  20*  of  the  sa*>le  variation  In 
award  fee  payaents  within  the  sample  of  Army  contracts. 
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estimates  and  to  estimate  the  parameters  for  the  population  of  award  » 
fee  contracts,  not  just  those  that  were  sampled,  but  time  limitations  pre- 
vent doing  so  at  this  point. 

This  result  supports  the  original  hypothesis  that  Increases  In  the 
fixed  fee  tend  to  act  as  a dls-lncentlve  for  the  award  fee.  Field  observa- 
tions Indicate  the  presence  of  a counter  factor  however.  Inexperienced 
contractors  frequently  prefer  and  sometimes  demand,  a large  fixed  fee.  As 
they  gain  experience  they  frequently,  but  not  always,  accept  lower  fees  on 
subsequent  contracts.  It  has  also  been  suggested  that  higher  fixed  fees 
ar\s  associated  with  higher  risk  contracts  where  the  risk  of  failure  would 
be  expected  to  be  higher.  The  possible  effect  of  these  two  factors,  con- 
tractors' experience  with  CPAF  contracts  and  risk  Is  unfortunately  unknown. 

4.  Variability  of  Award  Fee  Payments  Between  Contracts 


If  all  awards  are  the  same  then  there  should  be  no  difference 
between  the  level  of  award  fee  payments  between  contracts.  The  usual 
parametric  procedure  to  test  this  hypothesis  would  be  a one-way  analysis 
of  variance.  However,  Initial  Investigation  reveals  a drastic  departure 
from  the  underlying  necessary  assumption  of  homogeneous  variances.  A quick 
glance  at  Figure  3 reveals  a drastic  departure  of  Individual  awards  from 
a normal  distribution.  Therefore,  a non-parametrlc  (distribution  free) 
method  Is  used  to  Investigate  the  differences  between  contracts.  The 
Kruskal-Millls  one-way  analysis  of  variance  by  ranks  Is  an  extremely  useful 


test  for  deciding  whether  k Independent  samples  are  from  different  popu- 
lations and  Is  chosen  for  this  reason.  Sample  values  almost  Invariably 
differ  somewhat,  and  the  question  Is  whether  the  differences  among  the 
samples  signify  genuine  population  differences  or  whether  they  merely 
represent  expected  chance  variations  due  to  sampling.  The  Kruskal-Wallis 
technique  tests  the  null  hypothesis  that  the  k samples  come  from  the  same 
population  or  from  Identical  populations  with  respect  to  averages.  The 
test  assunes  only  that  the  variable  under  study  has  an  underlying  continuous 
distribution  and  1$  measured  on  at  least  an  ordinal  scale.  The  ten  con- 
tracts that  meet  the  requirements  (over  five  observations  within  each  group) 
were  analyzed  with  the  following  results: 

H * 39.669 

This  statistic  is  distributed  in  the  large  sample  case  (all  of  the 
requirements  for  which  have  been  met  by  our  application)  as  Chi-aquare  with 
k - 1 degrees  of  freedom.  Therefore  since; 

H . 39.669  > X2,  05,  9)  ■ 16.919 

and; 

> x2(.00l,  9)  ■ 2788 

It  Is  concluded  that  there  Is  a very  highly  significant  difference 
between  the  contracts  analyzed.  This  repudiates  the  contention  that  all 

contractors  get  the  same  level  of  award. 

_£•  * * * i-  ' 

5.  Variability  of  Award  Fee  Payments  Within  Contracts 

Some  observers  believe  that  award  fees  are  Ineffective  because 
they  are  all  the  same  and  that  a given  contractor  gets  about  the  smse 
award  In  successive  award  periods.  It  would  be  useful  to  test  this  theory 


between  contracts,  however,  to  do  so  would  require  data  of  successive 
similar  award  fee  contracts  with  Individual  contractors.  The  available 
data  will  not  support  such  an  analysis.  Therefore,  the  question  Is  addressed 
by  analyzing  the  behavior  of  awards  within  Individual  completed  contracts 
of  our  sample.  The  analysis  concentrates  on  the  differences  between  the 
Individual  awards  within  contracts.  Careful  Inspection  of  the  data  for 

a 

Individual  contracts  Indicate  a wide  diversity  of  such  differences  with  no 
discemable  pattern,  therefore  the  analysis  Is  directed  toward  an  overall 
effect  if  one  exists.  Before  proceeding  with  the  analysis,  the  distribution 
of  the  first  differences  between  Individual  awards  must  be  defined,  for 
that  may  strongly  influence  methods  that  can  be  used  for  the  analysis. 

a.  Distribution  of  First  Difference.  Data  from  the  sample  of 
completed  contracts  yielded  99  observations.  Each  observation  consists  of 
the  difference  between  Individual  awards  (percentage  of  potential  actually 
paid)  within  a given  contract.  Analysis  of  this  data  will  provide  Informa- 
tion about  the  "growth"  of  award  fees  within  contracts.  Contracts  with  only 
one  award  were  obviously  excluded  since  they  contain  no  such  data.  Contracts 
paid  on  the  billing  fee  principle  were  also  excluded  since  the  data  from 
them  would  distort  the  effect  as  these  fees  arc  later  adjusted.  The  data  set 
represents  the  variation  between  successive  awards  within  completed  contracts. 
Figure  4 displays  this  data  in  a histogram  together  with  the  normal  approximation 


FIGURE  4 DISTRIBUTION  OF  FIRST  DIFFERENCES 
BETWEEN  INDIVIDUAL  AWARDS  OF  COMPLETED  CONTRACTS 
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DISTRIBUTION  OF  FIRST  DIFFERENCES 
BETWEEN  INDIVIDUAL  AWARDS  OF  ONGOING  CONTRACTS 


to  it.  The  normal  approximation  shows  the  expected  distribution  for  a 
sample  of  the  same  size  and  with  the  same  mean  and  standard  deviation  as 
the  observed  data.  The  departure  from  normality  is  obvious.  In  order  to 
determine  If  such  a departure  Is  likely  attributable  to  chance  a Chi-Square 
test  of  goodness  of  fit  was  used.  The  results  are: 

«Vj)  ’ 40'597  > x2(.05.  8)  ■ ,5-507 

* x2(.01.  8)  ’ 20'M0 

This  indicates  a departure,  significant  at  the  1%  level,  from  a normal 
distribution.  Therefore,  the  sample  may  not  be  regarded  as  a random 
sample  drawn  from  a normal  distribution.  A close  inspection  of  Figure  4 
and  the  expected  versus  the  observed  frequencies  within  the  Chi-Square 
Table  (not  shown)  Indicate  a generally  symmetrical  design  with  an  "excess" 
of  observations  near  the  mean,  a "deficiency"  at  a medium  distance  from  the 
mean  and  the  expected  frequencies  at  the  extreme  deviation  from  the  mean. 
Because  of  the  nature  of  these  deviations  the  use  of  parametric  measures  Is 
justified  to  test  that  the  mean  Is  significantly  different  from  zero.  The 
large  sample  test  for  u based  on  the  Z statistic  Is  used  as  follows: 


45 


decision  criteria;  reject  null  hypothesis  (of  no  difference)  if  Z > Z '• 

The  results  are: 

Z ■ '-978  > z(.05)  ’ ’-64 

Therefore  the  conclusion  is  reached  that  the  mean  of  the  first  difference 
is  significantly  different  from  zero.  In  fact,  since  this  is  a "one  tail 
test"  we  may  conclude  that  the  mean  is  larger  than  zero  and  is  not  likely 
to  have  occurred  by  chance  alone.  This  indicates  cn  the  average  a tendency 
toward  higher  awards  as  the  contract  progresses.  This  is  an  overall  effect 
with  individual  contracts  showing  large  deviations  from  the  effect.  The 
influence  of  the  average  effect  which  is  assumed  to  be  linear  Is  shown  in 
Figure  5 where  the  effect  is  centered  on  the  level  of  award  fees  computed 
earlier,  and  extended  over  the  average  length  of  the  contracts  observed. 
Next,  the  absolute  value  of  the  first  differences  are  investigated. 

b.  Analysis  of  Absolute  Value  of  First  Differences.  It  was 
shown  above  that  the  changes  from  award  to  award  within  contracts  do  not 
cancel  out  but  result  in  a positive  value  significantly  different  from  zero. 
Therefore  they  are  not  an  accurate  estimator  of  the  next  award.  Of  addi- 
tional Importance  Is  the  absolute  value  of  the  differences.  These  values 
reflect  the  magnitude  of  the  changes  regardless  of  direction.  The  range  of 
the  absolute  value  of  the  first  difference  Is  from  less  than  one-tenth  of 
a percentage  point  to  over  28*.  The  average  is  5*  points.  This  gives  an 
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PERCENT  OF  AWARD  FEE  PAID 


FIGURE  5 AVERAGE  RELATIONSHIP  OF  AWARD 

FEE  PAYMENT  TO  STAGE  OF  CONTRACT 
COMPLETION  ESTIMATED  FOR  SAMPLE  DATA* 
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STAGE  OF  CONTRACT  PROGRESSION 


♦Trend  significant  at  the  95%  level  of  confidence 
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Indication  of  the  average  fluctuation  of  awards  within  Individual  contracts. 
The  relation  between  this  factor  and  the  level  of  the  Individual  contract 
Is  of  Interest  but  Is  not  Investigated  here. 

6.  Findings  from  Analysis  of  Ongoing  Contract  Data 

The  distribution  of  the  Individual  awards  for  ongoing  contracts 
and  the  first  differences  are  shown  In  Figure  3.  It  Is  Immediately  apparent 
that  the  variability  of  these  awards  as  well  as  the  differences  between 
successive  awards  Is  far  greater  than  comparable  data  from  the  sample  of 
completed  contracts.  It  Is  obvious  that  there  are  fewer  high  awards,  more 
lower  awards  and  the  bulk  of  awards  are  concentrated  at  a lower  level  than 
the  awards  for  completed  contracts.  As  would  be  expected  the  average  of 
the  awards  1$  much  lower  for  this  sample  at  67%.  This  data  base  will  be  used 
to  develop  a separate  estimate  of  the  growth  factor.  The  average  growth 
factor  for  the  ongoing  contracts  Is  1.16  which  is  very  close  to  the  value 
of  1.28  for  the  completed  contracts.  This  factor  Is  based  on  a smaller 
average  number  of  awards  per  contract  than  the  growth  factor  for  completed 
contracts.  The  relationship  of  these  estimates  Is  shown  In  Figure  6 where 
they  are  both  plotted  on  the  same  scale.  The  figure  shows  that  three  per- 
centage points  of  the  16  percentage  points  separating  the  level  of  payments 
for  the  completed  and  ongoing  contracts  is  accounted  for  by  the  growth 
factor  which  has  operated  for  a shorter  period  In  the  ongoing  contracts. 
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7.  Summary 

In  sumary,  the  Individual  award  payments  cover  a wide  range, 
fluctuate  frequently  and  unpredlctably  within  an  Individual  contract.  They 
behave  as  though  they  are  Independent  of  all  other  factors  Investigated 
except  the  structure  of  the  contract  Itself.  All  of  the  evidence  points 
to  the  conclusion  that  they  do|reflect  a subjective  assessment  of  performance 
and  not  other  extraneous  factors.  The  awards  are  definitely  not  relatively 
constant  or  nearly  all  the  same.  The  level  of  awards  for  contracts  cover 
a narrower  range  of  values  but  each  of  these  Is  an  average  of  several  awards 
which  obscures  the  diversity  of  Individual  awards.  However,  the  difference 
In  percentage  of  award  fee  paid  for  contracts  Is  statistically  significant 
showing  that  the  contracts  are  not  similar  In  level  of  payment. 
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CONCLUSIONS  AND  RECOMMENDATIONS 

A.  CONCLUSIONS 

1 . Level  of  Award  Fees 

It  Is  concluded  that  the  mean  current  level  of  award  fee  payments 
for  the  19  contracts  is  83%  of  the  maximum  available  award  fee  with  a 
standard  deviation  of  6%.  Based  on  the  DOD  Contractor  Performance  Evaluation 
and  the  NASA  Rating  System  for  CPAF  support  contracts  this  is  a reasonable 
and  desirable  level.  These  measurements  are.  In  effect,  field  assessments 
of  the  relative  success  of  each  contractor  In  meeting  his  distinctive 
goals;  they  are.  In  fact,  a field  assessment  of  effectiveness.  If  the  con- 
cept Is  successfully  applied  the  awards  should  be  high.  The  current  level 
reflects  overall  excellence,  but  not  outstanding  performance.  There  appears 
to  be  a decrease  In  award  fee  level  for  ongoing  contracts  that  should  be  a 
subject  for  continuing  research. 

2.  Procedures  for  Determining  Award  Fees 

Several  problems  have  been  previously  reported  In  the  Implementa- 
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tlon  of  award  procedures.  In  addition,  same  new  problems  that  have  been 
ufureported  were  uncovered. 

a.  Initial  Problems  Affecting  Effectiveness  of  Award  Fee 
Procedures.  In  the  Initial  stages  of  award  fee  contracting  many  pre- 
viously reported  problems  emerge  such  as,  establishing  the  correct 
organizational  and  personnel  level  for  the  award  fee  evaluation  board 
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and  the  fee  determining  official.  Problems  also  develop  with  communl- 
cations  both  within  the  government  and  between  the  government  and  con- 
tractor as  the  administrative  structure  and  procedures  develop. 

b.  Continuing  Problems  Affecting  Effectiveness  of  Award  Fee 
Procedures . Problems  resulting  from  the  general  nature  of  the  method  and 
different  philosophies  applied  under  widely  differing  circumstances 
continue.  They  revolve  around  the  basic  nature  and  structure  of  the 
evaluaton  (measuring)  system  and  the  methods  applied  In  developing 
the  scores  and  awards.  They  are  basically  problems  of  measurement, 
technique  and  philosophy  resulting  from  judgemental  evaluation. 

3.  Effectiveness  of  the  Award  Fee  Concept 

As  currently  applied  In  the  Army  the  award  fee  concept  is 
effective  and  would  earn  an  excellent  rating  based  on  either  the  DOD 
or  NASA  scoring  system.  It  Is  regrettable  that  It  was  not  practical  to 
develop  the  cost  side  of  the  cost-effectiveness  question.  If  dat*  could 
be  obtained  to  support  such  a detailed  analysis  a clearer  picture  of 
the  whole  process  would  emerge.  However,  In  view  of  the  difficulties 
encountered  with  that  approach,  this  report  concentrates  on  the  effective- 
ness portion  of  the  question.  Most  of  the  Initial  problems  that  are  quite 
prevalent  when  CPAF  contracts  are  first  employed  are  resolved  as  experience  Is 
developed,  administrative  procedures  and  practices  are  refined  and  as  communi- 
cations both  within  the  government  and  between  the  government  and  contractors  are 
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Improved.  Award  fees  appear  to  be  Independent  as  they  should  be  of 
several  contractual  factors  that  are  often  said  to  exert  an  undesirable 
Influence.  They  vary  widely  and  do  not  follow  any  recognizable  pattern 
except  for  an  expected  growth  factor.  There  are,  however,  some  problems 
that  continue  to  exist  and  detract  from  achieving  maximum  effectiveness  of  the 
award  fee  system. 

a.  External  Effects  Affecting  Effectiveness  of  the  Award 
Fee  Concept.  Several  external  contractual  factors  were  studied  and 
not  found  to  detract  from  the  effectiveness  of  the  award  fee  concept 
as  currently  applied  In  the  field.  Many  of  the  suppositions  concerning 
these  factors  appear  to  stem  from  a lack  of  knowledge  of  the  detail 
workings  of  the  system.  Little  previous  study  has  been  directed  In 
this  area. 

4.  Remaining  Problems 

Problems  remain  in  making  subjective  evaluations  and  combining 
them  Into  numerical  scores  which  are  then  converted  to  award  fees.  A 
problem  of  bias  was  discovered  making  It  very  unlikely  that  a contractor 
could  obtain  the  maximum  award  fee  under  the  existing  system. 

Somewhat  different  procedures  and  philosophies  are  employed  In  different 
commands  making  It  unlikely  that  many  people  gain  from  all  of  the 
experience  available  within  the  total  system.  The  dynamic  nature  and 
uniqueness  of  Individual  applications  make  different  philosophies 
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necessary  but  also  prohibit  the  development  of  a practical  theory  for 
guidance. 

a.  Additional  Research  Needed.  Little  factual  Information 
exists  providing  an  overall  picture  of  the  operation  of  the  award  fee 
contracting  system.  Most  of  the  knowledge  gained  has  been  limited  to 
the  experience  from  a few  contracts  or  within  a command.  The  interchange 
of  Ideas  and  lessons  learned  throughout  the  Amy  would  be  very  useful. 
Research  Is  needed  to  develop  a more  effective  method  of  combining  subjective 
evaluations  Into  overall  scores.  Research  into  coamand  differences  and 
the  apparent  decline  in  award  level  for  on-going  contracts  would  be 
helpful  In  providing  a factual  basis  for  more  effective  decisions  in 
future  award  fee  contracting.  Also  needed  is  a better  view  of  the 
unique  administrative  costs  involved  In  award  fee  contracting. 

B.  RECOtiEWDATIOMS 

As  a result  of  this  study,  the  following  recommendations  are  made: 

1.  Continue  use  of  award  fee  contracts. 

2.  Continue  present  guidance  until  research  and  feedback  from 
field  experience  provides  the  factual  basis  for  Improvement. 

3.  Continue  research  concentrating  In  the  following  areas: 

a.  Measurement  procedures  and  evaluation  plans. 

b.  The  apparent  changing  level  of  award  fee  payments. 

c.  Differences  In  application  of  award  fee  contracts  between 
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4.  DARCOM  establish  an  award  fee  policy  to  Improve  the  Interchange 
of  Ideas/experience  and  to  disseminate  the  results  of  research  In  the 
area  of  award  fee  contracts. 

a.  Establish  within  DARCOM  an  Individual  to  serve  as  a point 
of  contact  to  Improve  Intercommand  communication. 

b.  Sponsor  periodic  workshops  for  people  actively  engaged  In 
developing  and  administrating  award  fee  contracts  to  aid  In  the 
dissemination  of  experience  gained  locally  throughout  the  Army. 
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APPENDIX  A 

DESCRIPTION  OF  SAMPLE  DATA  FOR  COMPLETED  CONTRACTS 

Basic  data  in  the  sample  of  completed  contracts  consists  of  the 
estimated  cost  for  the  Initial  contract  and  estimated  cost  for  the 
completed  contract  and  the  amount  of  associated  fees.  The  fees  are 
broken  down  Into  fixed  fee  and  the  maximum  award  fee  allocated  for 
that  contract  (award  fee  pool)  as  envisioned  at  the  beginning  of  the 
contract  and  again  at  contract  completion.  These  two  points  contrast 
what  was  initially  planed  with  what  actually  happened.  In  addition, 
the  award  fee  that  was  actually  paid  during  the  life  of  the  contract 
was  obtained  In  order  to  compute  the  percentage  of  the  award  fee  that 
was  actually  paid.  For  the  purpose  of  this  study  a contract  Is  con- 
sidered completed  when  the  final  cost  and  fees  have  been  established. 
Additional  data  were  collected  on  the  amount  and  dates  of  Individual 
awards  and  the  dates  of  contract  beginning  and  ending.  These  factors 
are  not  discussed  here  but  are  brought  Into  play  elsewhere  In  the  analysl 
Summary  statistics  describing  the  basic  data  are  presented  In 
Table  A-l , (note  that  the  estimated  costs  are  expressed  In  millions 
of  dollars  and  the  fees  are  expressed  in  thousands).  The  sample  mean 
Is  used  to  describe  the  central  tendency  and  the  usual  measures  of 
dispersion  about  the  mean,  the  standard  deviation  and  the  standard 
error  are  used  to  describe  the  variability.  Caution  should  be  exercised 


TABLE  A-l  SAMPLE  MEASURES  OF  CENTRAL  TENDENCY  AND  DISPERSION  (BASE  VALUES) 
MEASURE  OF  CENTRAL  TENDENCY  - COMPLETED  CONTRACTS 


In  drawing  the  usual  conclusions  from  only  those  measures,  however. 

To  assist  In  a more  accurate  assessment  of  the  basic  data  the  highest 
and  lowest  value  observed  In  the  sample  and  the  range  between  the 
two  are  also  noted  In  Table  A-1.  In  analyzing  any  of  the  coats  or 
fees  summarized  here  It  Is  obvious  that  the  mean  does  not  fall  at 
about  the  midpoint  of  the  range  of  highest  and  lowest  values.  In 
fact.  In  all  cases,  the  mean  falls  much  closer  to  the  lowest  value  than 
the  highest.  The  practical  meaning  of  this  phenomlna  Is  that  a few 
large  contracts  are  heavily  Influencing  the  results  because  the  distri- 
bution of  costs  and  fees  is  strongly  skewed  to  the  right. 

Key  values  computed  from  the  basic  sample  data 

From  the  dollar  values  of  contracts,  costs  and  fees  just  described 
In  the  previous  section,  several  key  percentage  factors  were  computed 
which  relate  the  specific  fees  to  their  associated  contract  costs.  A 
suanary  of  this  derived  data  Is  presented  In  this  section  In  Table 
A-2.  As  these  data  are  the  elements  of  the  fee  structure,  the  heart 
of  an  award  fee  contract,  they  should  be  of  considerable  Interest  to 
contracting  officers,  contract  specialists  and  others  directly  Involved 
In  award  fee  contracting.  In  order  to  aid  In  Interpreting  the  data 
presented  In  Table  A-2  a brief  discussion  follows. 

Individual  values  fbr  each  factor  were  computed  for  each  Individual 
contract  using  the  appropriate  dollar  value  of  cost  and  fees.  The 
percent  of  maximum  award  fee  actually  awarded  here  applies  to  the 


indicators  of  the  true  dispersion. 
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total  contract.  It  shows  the  award  fee  that  was  actually  earned  during 
the  life  of  the  contract  as  a percent  of  the  maximum  that  the  contractor 
theoretically  could  have  earned  had  he  performed  perfectly  and  been 
paid  accordingly,  this  measure  can  only  be  calculated  after  the  con- 
tract has  been  "completed." 

The  mean  of  the  individual  contract  values  along  with  the  associated 
standard  deviation  and  standard  error  are  computed  In  the  usual 
way  and  presented  in  Table  A-2.  As  was  pointed  out  earlier,  caution 
Is  required  in  interpreting  these  statistics.  The  very  large  standard 
deviations  in  relation  to  the  range  of  values  observed  and  the  location 
of  the  mean  in  relation  to  the  highest  and  lowest  observed  values  Indicate 
that  the  usual  assumption  of  an  underlying  normal  distribution  does  not 
hold  exactly.  The  only  exception  Is  the  percentage  of  maximum  award 
fee  actually  awarded.  For  these  reasons  a derived  mean  developed  from 
the  sample  totals  is  presented  as  a more  useful  descriptor  of  the  sample 
data.  This  procedure  has  the  same  effect  as  weighting  each  Individual 
contract  value  by  the  contract  size  (appropriate  dollar  value)  but  was 
selected  for  computation  to  reduce  rounding  errors,  (note  that  the 
two  measures  of  central  tendency,  the  weighted  mean  and  the  unweighted 
mean,  give  Identical  results,  except  for  rounding  errors.  In  the  case 
of  the  percentage  of  award  fee  awarded).  The  highest  and  lowest  observed 
values  and  the  range  between  them  Is  also  displayed  as  an  aid  to  betted 
understanding  of  the  properties  of  the  data  base. 
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